clarify range of replication of excluded group feed #15

staltz · 2023-02-24T13:29:25Z

For issue #9

Powersource

Looks pretty good but would be nice to bikeshed for 5 min

Powersource · 2023-02-24T15:00:53Z

README.md

+  * 4.1.4.B. `excludes` is an array of objects with the shape
+  `{id, groupFeedId, sequence}` where `id` is the root metafeed ID of `c`,
+  `groupFeedId` is `Gc`'s ID, and `sequence` is the sequence number of the last
+  message `a` possesses from `Gc`.  In this case `c` is the only excluded
+  member, so we only have one `{id, groupFeedId, sequence}` object but Section
+  4.1. also supports excluding multiple members at once.


We should be able to (i think?) calculate this technically by looking at the group tangle but I guess this way is both clearer and more efficient.

A really fun bug would be if the excluder starts creating this message, but then inbetween that and creating the tangle for the message, receives another message from the excluded person, pointing the group tangle at that message in the tangles.group.previous. Then the announce of this sequence and the group tangle would disagree on which messages to include.

I guess a way of preventing that would be for the excluder to stop replicating the excluded person before starting to create this message.

Minus the tangle thing, I wouldn't see a problem. The admin saw "the last straw" from Oscar and decided that Oscar has to leave. It doesn't matter if Oscar keeps on posting because "the last straw" message is the one that the admin decided is the last one.

But since you mentioned tangles and their linking, and since I don't understand tangles (I thought I did), I'm like 🤷 and need someone to explain like I'm five. Maybe @mixmix ?

yeah but if we later on add e.g. tangle sync, then people using that would end up with a different set of messages than people syncing whole feeds.

either way this is just in case an implementation has what's essentially a bug.

maybe just add a warning, something like

"implementers SHOULD (must?) be careful to not include newer messages than sequence in any tangle."

@staltz did you get to understanding tangles? did my scuttelbutt write-up help?

Hmm... if the tangle thing is gonna be used/ useful we have an interesting edge case:

Staltz excludes { id: Oscar, groupFeedId: OscarG, sequence: 234 }

Oscar published message in OscarG with sequence: 235

Mix replicates Oscar, and extends the group tangle off of that message of Oscars, then replicates Staltz and is like "oops I included a message after the exclusion in the group tangle").

We aren't using the tangle ordering (tangles.group.previous) thing yet for anything... so this is quite hard to design for. We mainly use tangles.group.root to find messages in a particular group....

Gut feeling is say "fuck it" and just continue on.... but if people receive Staltz's exclusion message first they will stop replicating and then have a gap in their tangle... hmmm.
Hard

No, in this situation

Mix replicates Oscar, and extends the group tangle off of that message of Oscars, then replicates Staltz and is like "oops I included a message after the exclusion in the group tangle").

even without tangles, Mix might be replying to Oscar's latest comment. But Staltz would never see that comment since he doesn't want to go beyond the sequence he announce before. This would break discussion threads. Tangles in this case doesn't create the problem, it's a method of dealing with the problem that is eventual consistency.

Mix might be replying to Oscar's latest comment. But Staltz would never see that comment

I could see Mix's comment, I just won't see what did Oscar say that Mix is replying to. I think that's an okay compromise to make.

By allowing Oscar to be "eventually excluded", we open up a few abuse channels too, because Oscar knows he's excluded but still has his final words to make, and at that point he might not give a shit about the code of conduct anymore. I think that's a worse outcome than discussion threads looking slightly punched with holes.

We already don't make much of/any effort to deal with bad actors once they're inside a group. And not accomodating for eventual consistency would break a core design choice of scuttlebutt. It's not necessarily "slightly punched with holes", it might as well be long discussions. It'd also break i think one of our design principles for groups, that everyone should have the same set of messages etc. That's e.g. the reason we ignore blocks in groups.

@mixmix @arj03 Help us out here? We're stuck

mixmix · 2023-03-15T01:40:43Z

here is a visualisation of the problem

flowchart RL

classDef default stroke:none;
classDef cluster fill:#fff,stroke:#333;
linkStyle default stroke:#333,stroke-width:0.5;

subgraph Alice
A:5---A:4---A:3---A:2---A:1
end
classDef A fill:#3A86FF,color:#fff;
class A:1,A:2,A:3,A:4,A:5 A

subgraph Bob
B:3---B:2---B:1
end
classDef B fill:#8338EC,color:#fff;
class B:1,B:2,B:3 B

subgraph Oscar
direction RL
O:3---O:2---O:1
end
classDef O fill:#FFBE0B,color:#fff;
class O:1,O:2 O

%% exclude
A:4-."(exclude)".-xO:2:::lastMsg

%% group tangle
A:2-->A:1
B:1-->A:2
O:1-->A:2
O:2-->O:1
B:2-->B:1 & O:2
A:3-->B:2
A:4-->A:3
O:3-->A:3
B:3-->O:3
A:5-->A:4 & B:3

linkStyle 9,10,11,12,13,14,15,16,17,18,19,20 stroke:#fb5607,stroke-width:3.5;

Key:

[A:2] = feed A, sequence 2
--- a backlink
- grey --- = the feed "tangle" (previous)
- orange --- = the groups tangle

In this diagram, we have several people contributing to a group tangle.
Then Alice decides to exclude Oscar (message "A:4" says "exclude Oscar after O:2").

A problem arises in that Bob was out of contact and has extended the group feed off of O:3...which should have been excluded... then Alice (who has stopped replicating Oscar) has B:3 but not O:3 which is part of the tangle now.....

mixmix · 2023-03-15T01:51:31Z

this diagram is cool because you can see the partial ordering the group tangle is providing.

Here's what it looks like if you're Alice and don't replicate Oscar:3, and are trying to see the group tangle - you can see B:3 references something, but not where it is.
NOTE current ssb ids don't include anything about this author so you wouldn't even know if the message is Oscar's (we should change this)

flowchart RL

classDef default stroke:none;
classDef cluster fill:#fff,stroke:#333;
linkStyle default stroke:#333,stroke-width:0;

subgraph Alice
A:5---A:4---A:3---A:2---A:1
end
classDef A fill:#3A86FF,color:#fff;
class A:1,A:2,A:3,A:4,A:5 A

subgraph Bob
B:3---B:2---B:1
end
classDef B fill:#8338EC,color:#fff;
class B:1,B:2,B:3 B

subgraph Oscar
direction RL
O:3---O:2---O:1
end
classDef O fill:#FFBE0B,color:#fff;
class O:1,O:2 O
linkStyle 6 stroke-width:0;


%% exclude
A:4-.-xO:2:::lastMsg
linkStyle 8 stroke-width:0;


%% group tangle
A:2-->A:1
B:1-->A:2
O:1-->A:2
O:2-->O:1
B:2-->B:1 & O:2
A:3-->B:2
A:4-->A:3
%% O:3-->A:3
B:3-->O:3
A:5-->A:4 & B:3

linkStyle 9,10,11,12,13,14,15,16,17,18,19 stroke:#fb5607,stroke-width:3.5;

Questions

should A:5 backlink to B:3 (as shown), given it knows Bob is part of the group tangle.
Is this DAG still good/ useful for ordering?
should we give up on partial ordering?

mixmix · 2023-03-15T02:11:10Z

because the group tangle is only for ordering, and is not currently carrying any logic, I think the above is fine.
Basically it would mean we just need to define "previous" as :

get all the group messages
build a graph
the "previous" is the "tips" of that graph
- a) we include any disconnected nodes here as "tips" to weave them back in (it's ok if this tangle is "frayed"

mixmix · 2023-03-15T02:13:07Z

Another noticing is that B:3 follows B:2, and we know Bob is part of the group, so it's chill when you find an "O:3" that you don't know about to substitute an alternative backlink we know is valid "B:2"

I think we should just munge it in and see what happens. Probably fine

staltz · 2023-03-15T08:46:06Z

Great, thanks for the diagrams and the thinking, I like it.

NOTE current ssb ids don't include anything about this author so you wouldn't even know if the message is Oscar's (we should change this)

This is easy to change, we just need to use [authorId, sequence] instead of msgId inside the tangles structure. Given a msgVal, it should be trivial to construct that by just plucking msgVal.author and msgVal.sequence, unlike msgId which requires calculating the hash.

I also think this problem is something we can live with. I think it's more important that excluded members are excluded "effective immediately" instead of waiting for all peers to converge on it, which may not even ever happen, if one of the members is offline/dead and never coming back.

Powersource · 2023-03-15T12:24:11Z

we just need to use [authorId, sequence] instead of msgId

I guess we could change this but isn't this better as a separate change, that we do in general to URIs?

I also think this problem is something we can live with.

Why, when the problem is so easy to avoid? It's just some small extra logic in tribes2 needed do to it the tangle way. And the tradeoff when doing sequence instead of tangle is that we scrap eventual consistency and offline first. If we decide to scrap that then there's probably a lot of other corners we could cut in ssb.

re excluded "effective immediately" instead of waiting for all peers to converge on it

They will be excluded immediately (or for every individual as soon as they receive the exclusion message, not before then)

mixmix · 2023-03-16T01:55:50Z

@Powersource he staltz was talking about using [authorId, sequence] in tangles.group.previous (instead of the messageId), not for the exclude. This is just an alternative way of defining a messageId which I think we are likely to want to move towards generally.

Lets discuss this on call if this is feeling confused

clarify range of replication of excluded group feed

978c11c

staltz requested a review from Powersource February 24, 2023 13:29

Powersource reviewed Feb 24, 2023

View reviewed changes

Powersource mentioned this pull request Mar 2, 2023

Base replication stop on group tangle #19

Closed

This was referenced Sep 28, 2023

Specify how replication of excluded is stopped #34

Merged

Remove group tangles #35

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

clarify range of replication of excluded group feed #15

clarify range of replication of excluded group feed #15

staltz commented Feb 24, 2023 •

edited by Powersource

Loading

Powersource left a comment

Powersource Feb 24, 2023

Powersource Feb 24, 2023

staltz Feb 24, 2023

Powersource Feb 24, 2023

mixmix Feb 28, 2023

Powersource Mar 1, 2023

Powersource Mar 2, 2023

staltz Mar 7, 2023

Powersource Mar 7, 2023

staltz Mar 7, 2023

mixmix commented Mar 15, 2023

mixmix commented Mar 15, 2023

mixmix commented Mar 15, 2023

mixmix commented Mar 15, 2023

staltz commented Mar 15, 2023

Powersource commented Mar 15, 2023

mixmix commented Mar 16, 2023

clarify range of replication of excluded group feed #15

Are you sure you want to change the base?

clarify range of replication of excluded group feed #15

Conversation

staltz commented Feb 24, 2023 • edited by Powersource Loading

Powersource left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mixmix commented Mar 15, 2023

mixmix commented Mar 15, 2023

mixmix commented Mar 15, 2023

mixmix commented Mar 15, 2023

staltz commented Mar 15, 2023

Powersource commented Mar 15, 2023

mixmix commented Mar 16, 2023

staltz commented Feb 24, 2023 •

edited by Powersource

Loading